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ABSTRACT 


The  logic  of  a  technique  for  employing  technician  "confidence  that  a  defect 
exists"  for  maximizing  the  probability  of  malfunction  recognition  is  described.  The 
technique  is  based  on  and  drawn  from  parallel  thinking  in  signal  detection  theory. 
Operator  characteristic  curves  are  derived  for  a  variety  of  distributions  of  "confi¬ 
dence.  "  Continuous  and  discrete  distributions  of  "confidence"  are  considered  as 
well  as  single  and  double  criterion  levels.  The  implications  of  the  work  for  train¬ 
ing  and  posttraining  performance  evaluation  are  pointed  out. 
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CHAPTER  I 


INTRODUCTION 

Previous  research,  completed  by  Applied  Psychological  Services,  has  attempted 
to  set  into  focus  methods  for  estimating  personnel  subsystem  reliability.  These  studies, 
drawn  from  and  based  on  a  series  of  investigations  into  "personnel  psychophysical"  re¬ 
lationships,  sought  to  establish  techniques  for  quantitatively  assessing  the  probability 
of  successfully  performing  a  given  avionic  maintenance  act.  In  the  first  of  these  studies 
into  personnel  subsystem  reliability  (Siegel  &.  Pfeiffer,  1966),  the  utility  of  the  ratio  of 
effective  to  effective  plus  ineffective  performances  was  investigated  as  the  basic  in¬ 
gredient  for  personnel  subsystem  reliability  estimation.  Measures  of  effective  and  in¬ 
effective  performance  were  obtained  for  Fleet  avionic  maintenance  personnel  through 
magnitude  estimation  methods.  The  measures  were  anchored  to  a  number  of  avionic 
job  dimensions  isolated  through  a  multidimensional  scaling  analysis  of  the  avionics 
maintenance  job.  When  compounded  through  techniques  which  are  analogous  to  those 
employed  in  traditional  electronic  reliability  prediction,  the  evoked  numerics  were 
found  to  discriminate  between  squadrons  known  to  be  of  different  competency  by  other 
criteria  and  to  discriminate  within  squadrons  in  accordance  with  reasonable  expecta¬ 
tion.  Siegel  and  Pfeiffer  concluded  that: 

The  obtained  avionic  pe-sonnel  subsystem  (reliability) 
indices  seem  to  be  useful  for  posHraining  performance  apprais¬ 
al,  personnel  placement,  and  squadron  evaluative  purposes. 

Because  of  the  reasonableness  of  the  results,  the  ease  of  employment,  and  the 
general  utility  of  the  approach  employed  by  Siegel  and  Pfeiffer,  Siegel  and  Miehie  (1967) 
extended  the  work  to  consider  a  number  of  circumstances  not  originally  considered.  Ad¬ 
ditionally,  a  measure  of  "effectiveness,  "  as  separate  from  "reliability,  "  was  derived. 


The  effectiveness  calculation  considered  net  only  the  reliability  of  the  individual  tech¬ 
nicians  performing  the  maintenance  acts,  but  also  elapsed  time,  amount  of  manpower 
employed,  and  job  activity  repetition.  It  was  contended  that  the  technique  is  useful  for: 

2.  quantitative  comparison  of  the  effectiveness  of  different  teams 
or  individuals  who  perform  the  same  task 

2.  prediction  of  the  performance  effectiveness  of  a  team  or  of  an 
individual  on  a  task 

3.  derivation  of  training  requirements 

4.  optimization  of  personnel  assignments  and  maintenance  pro¬ 
cedures 

5.  evaluating  the  design  of  new  systems  or  alternative  designs 
of  the  same  system 

Additionally,  Siegel  and  Miehle  developed  nomographs  and  tables  to  simplify  the  com¬ 
putational  aspects  of  the  procedures  they  described. 

Purpose  of  Present  Report 

The  previous  studies  consider  only  peripherally  the  trouble-shooting  aspect  of 
avionic  maintenance.  Within  the  technique  described,  the  job  activity,  "electro  cogni¬ 
tion,  "  is  considered  to  encompass  a  host  of  mental  acts  involved  in  the  maintenance  and 
trouble-shooting  of  avionic  equipment.  Yet,  it  is  known  that  malfunction  recognition 
represents  a  particularly  troublesome  aspect  of  the  maintenance  procedure,  whether 
periodic  inspections  or  malfunction  diagnosis  and  location  are  involved.  Good  com¬ 
ponents  are  often  erroneously  replaced  and  marginal  or  defective  components  over¬ 
looked. 

As  the  logical  extension  of  th  *  previous  work  in  "personnel  psychophysics,  "  it 
seemed  reasonable  to  apply  the  psychophysical  developments  of  signal  recognition  theory 


to  the  malfunction  recognition  issue.  The  centra)  problem  then  becomes  that  of  demon¬ 
strating  a  quantitative  method  for  determining  the  probability  that  an  avionic  technician 
will  say  that:  (1)  a  defect  is  present  when  a  defect  is  in  fact  present,  and  (2)  a  defect  is 
present  when  in  fact  no  defect  is  present. 

The  methods  to  be  described  in  subsequent  chapters  of  this  report  are  analogous 
to  those  used  by  Swets  and  his  coworkers  (cf. ,  1964).  In  the  present  report,  the  confi¬ 
dence  that  a  defect  exists  replaces  the  perceived  intensity  of  a  signal  in  their  work. 


CHAPTER  H 


METHODS 

In  certain  avionic  maintenance  activities,  satisfactory  performance  may  be  said 
to  occur  when  action  i3  taken  when  it  should  be  taken,  and  no  action  is  taken  when  no 
action  is  warranted.  Examples  of  this  occur  during  routine  equipment  inspection,  fault 
location  and  isolation,  and  malfunction  correction.  Let  us  consider  all  of  these  under 
the  generic  classification  of  defect  recognition. 

Defect  Recognition 

Let  x  be  a  measure  of  the  maintenance  technician's  confidence  that  a  defect  ex¬ 
ists,  with  0<x<  1.  For  the  situation  in  which  the  technician  has  complete  confidence 
that  a  defect  exists,  x  =  1.  When  the  technician  has  complete  confidence  that  no  defect 
exists,  x  =  0.  Intermediate  confidence  values  may  exist  because  of  incomplete  informa 
tion,  conflicting  information,  inadequate  knowledge  about  the  equipment  system,  etc. 

Due  to  this  uncertainty,  a  technician  may  have  different  values  of  confidence  on 
different  occasions,  even  though  the  situations  presented  to  him  are  the  same.  This 
variation  is  representable  by  a  confidence  distribution. 

When  there  is  a  defect,  confidence  that  a  defect  exists  is  high  (x  is  near  1)  most 
of  the  time.  When  there  is  actually  no  defect,  in  most  cases  x  is  low. 


5. 


Continuous  Distribution  of  Confidence 


For  a  continuous  distribution  of  x  (the  measure  of  confidence  that  a  defect  ex¬ 
ists),  let  f(x)  be  the  frequency  or  probability  density  function.  When  a  defect  exists, 
this  function  is  monotonicaliy  increasing,  having  a  maximum  value  for  x  =  1  (Figure  1). 
When  a  defect  does  not  exist,  we  have  a  different  frequency  function  which  is  monotoni- 
cally  decreasing.  It  has  a  maximum  for  x  =  0  (Figure  2). 


I  x 


Figure  1  Hypothetical  frequency  function 
when  defect  exists 


I  x 


Figure  2  Hypothetical  frequency  function 
when  there  is  no  defect 


Note  that  J_*  f(x)dx  =  1,  i.  e. ,  the  area  under  the  curve  is  unity.  Since  0  <  x  <  1, 

/'  f(x)dx  -  i  in  our  case, 
o 

The  probability  that  the  measure  of  confidence  is  less  than  or  equal  to  a  con¬ 


stant,  k,  is:  Pr[x  5  kj. 


Pr[x  s  k]  =  /_Bf(x)dx 


6. 


This  probability  is  the  area  under  the  curve  of  f(x)  to  the  left  of  x  =  k.  The  distribution 
function  is  F(x)  =  /Xf(u)du  and  is  the  area  under  the  curve  of  f(x)  to  the  left  of  x.  The 

«.C0 

probability  that  a  <  x  <  b  is  P  [a  <  x  <  b]  =  /bf(x)dx  =  F(b)  ~  F(a). 

X*  cL 

Note  that  F'(x)  =  f(x).  Suppose  that  a  technician  will  report  a  defect  (will  say 
"yes")  when  his  confidence  that  there  is  a  defect  is  greater  than  or  equal  to  some  sub¬ 
jective  criterion  level,  c,  i.  e. ,  xi  c.  Then  he  will  also  report  that  a  defect  is  not 
present  (will  say  "no")  when  x  <  c. 

For  either  a  defect  or  no  defect,  the  probability  that  a  technician  will  say  "yes" 
is  Pr[x  Jc]=  J^f(x)dx.  When  there  actually  is  a  defect,  then  Pr[x  >c]  is  the  prob¬ 
ability  of  a  correct  response;  call  this  P^  [yes  [D].  When  there  is  no  defect,  then 
Prfx>cl  is  the  probability  of  an  incorrect  response  (false  alarm);  call  this  Pf  [yes  | D ']. 

Probability  of  Defect  Recognition 

For  either  a  defect  or  no  defect,  the  probability  that  a  technician  will  say  "no"  is 
P  [x  <  c]  =  /  f(x)dx.  When  there  actually  is  a  defect,  then  P  [x<  c]  is  the  probability 
of  an  incorrect  response  (failure  to  detect).  Call  this  Pf  [no|0].  When  there  is  no  de¬ 
fect,  then  P^  [x<  c]  is  the  probability  of  correct  response.  Call  this  Pf  [no  J D ' ]. 

ket  the  frequency  function  when  defect  exists  be  fj  and  for  no  defect  be  f0 . 

Then:  P  [yes|D]  =/‘  fj  (x)dx,  P  [yes|D']  =  fjfo  (x)dx,  P  [nojD]  =  J"L fj  (x)dx,  and 
Pr[no|D']  *  /ofo(x)dx. 

The  probability  that  the  subject  says  "yes"  and  there  is  a  defect  is:  Pf  [yesA  D]  * 
P^  [yes  |D]Pr[D],  The  probability  that  the  answer  is  "no"  and  there  is  no  defect  is 
Pr  [noAD'j  •  Pr[no|D']  P^D1]. 


The  probability  of  a  correct  response  {>  of  correct  responses),  R,  is: 


R  =  Pr  [yesA  D]  +  Pr  [noAD1] 

=  Pr[yes|D]Pr[D]  +Pr[no|D'lPr[D*] 

=  P r[D]/‘f1  (x)dx  +  Pr[D']/Cf0  (x)dx.  (1) 

i  v  I  0 

The  probability  of  an  incorrect  response  is: 

P  fyesAD']  t  P  [noAD] 
r  r 

=  Pr[yes|D']Pr[D']  +  Pr[nojD)Pr[D] 

»  P  (D]/ofi  (x)  dx  +  P  [D']  /‘fo(x)dx.  (2) 

r  r  c 

The  sum  of  these  probabilities  (correct  and  incorrect  response)  is  unity. 
These  relationships  can  be  alternatively  represented  as  indicated  in  Exhibit  I. 


Yes 

No 


D  D1 


Pr[yesAD] 

PytyesAD'] 

Pr(noAD) 

Pr(noAD'] 

Exhibit  I 

Exhibit  I  can  be  considered  as  a  Venn  diagram  in  which  the  large  rectangle 
represents  the  universal  set  of  all  logical  possibilities.  The  column  labeled  D  repre¬ 
sents  the  set  of  situations  in  which  a  defect  exists.  The  row  labeled  "Yes"  represents 
the  set  situations  in  which  "yes"  was  the  response.  The  cells  represent  intersections 
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8. 

of  the  sets  represented  by  rows  and  columns.  Their  measures  are  the  joint  prob¬ 
abilities,  whereas  the  measures  of  the  rows  and  columns  are  the  marginal  prob¬ 
abilities.  Note  that  only  if  independence  exists  does  P^fyesAD]  =  P^fyes] P^fD], 

This  situation  is  a  special  case  of  the  formulas  given  earlier. 

Reliability  and  Optiaua  Confidence  Criterion  Level 

Reliability  R  (or  the  probability  of  correct  response)  is  a  function  of  Pr[D]  and 
the  confidence  distribution: 

R  =  P  [D]/1  fi  (x)dx  +  P  [D'l /Cf0  (x)dx 
r  c  r  o 

HR 

For  maximum  R,  we  set  t—  =  0  to  find  the  optimum  value  of  c.  Assume  P  [D] 

dc  r 

is  constant.  The  implicit  equation: 

~  s  -P^JD]  fj (c)  +Pr[D'jf0(c>  -  0 

gives  optimum  value(s)  of  c. 

f,(c)  _  Pr[D']  _  1  -  PHD] 

ro  PFtCT  . Prig]' 

Suppose  that,  in  the  long  run,  for  an  unchanging  situation,  the  fra  lion  of  times 
a  defect  occurs  is  r.  Then  Pr(D]  *  r  andPr(D']*  1  -  r,  so  that  For 

example,  if  a  defect  occurs  20%  of  the  time  *  4.  Then,  for  maxi- 

mum  reliability,  the  technician  should  (and  probably  will)  make  c  such  that  *  4. 

As  example  1,  assume  the  graphs  of  f«  and  fi  as  shown  in  Figure  3. 


I  X 


Figure  3  Linear  frequency  functions,  with  (fj )  and  without  (fo )  defect 

(first  example) 


Note  that  the  vertical  scale  has  been  adjusted  so  that  the  area  under  each 

f  (c)  2c 

curve  is  equal  to  1.  Here,  fo(x)  =  2  -  2x,  and  fi  (x)  =  2x,  so  that  ^  ^  =  2^2c  ' 

2c 

For  maximum  reliability,  for  Pr[D]  =  .  2,  —  ^  -  4,  or  c  -  .  8. 

In  this  case,  the  technician  should  report  a  defect  when  his  confidence  value  is 
.  8  or  above.  For  any  Pr[D]  =  r. 


2c  s  1  -  r 
2-2c  r  ' 


Solving  for  c  gives:  c  =  1  -  r.  When  only  a  few  defects  exist,  r  is  low  and  con¬ 
sequently  c  is  large,  1.  e. ,  the  subject  will  or  should  say  "yes"  only  when  he  is  very 
sure  that  a  defect  exists. 

As  another  example,  consider  Figure  4.  The  area  under  each  curve  is  still  1. 


4  e 


Figure  4  Linear  frequency  functions,  with  <fi  >  wtd  without  (fo)  defect 

(second  example) 


f0 (x)  =  5-12,  5x  and  f  ,(x)  =  12.  5x  -  7.  5 


10. 


On  the  basis  of  the  distribution  shown,  when  a  defect  exists,  the  technician 
always  has  a  confidence  level  of  .  6  or  greater.  When  no  defect  exists,  the  technician 
always  has  a  confidence  of  .  4  or  less.  Therefore,  if  his  threshold  or  critical  value  c  is 
between  .4  and  .6,  he  will  always  be  correct  (R  =  1).  This  should  be  independent  of 
Pr[D].  Let  us  compute  R  for  .  4  <  c  <  .  6. 

R  =  Pr[D]/^fi  (x)dx  +  Pr[D'J /ofo(x)dx 
=  Pr[D](l)  +  Pr[D']  (1)  -  1 


Ifc  <  .4,  the  technician  will  sometimes  report  a  defect  when  there  is  none.  If  c  >  .6, 
the  subject  will  sometimes  fail  to  notice  a  defect. 

Consider  the  equation  for  optimum  c: 


Id£>  .  llil 
foTcT  r 

When  solved  for  c,  this  gives: 


or 


12,5c  -  7.5 
5  -  12.  5c 


C  *  .  4  +  .  2r. 


As  r  varies  from  0  to  1,  c  varies  from  .  4  to  .6.  In  this  example,  it  turns  out 
that  all  values  of  c  in  that  interval  are  optimum. 

Let  us  now  calculate  the  reliability,  R,  for  the  example  in  which  the  curves 
intersected  (first  example): 

R  *  r/*  2xdx  ♦  (1  -  r)  Jo  (2  -  2x>dx 

c 

-  r(x*  Jj.  ♦  (l-r)(2x  -  x*]o 
■  r(l  -  c1)  +  (1  -  r)(2c  -  c*  -  0] 

2  r  ♦  2c  -  c1  -  2rc. 


«“*■**• 


For  optimum  defect  recognition  (c  =  1  -  r)  this  becomes; 


R  =  r2  -  r  +  1 
m 

n  is  the  maximum  value  of  reliability  for  the  given  confidence  frequency  functions 
m 

and  a  given  r.  See  Figure  5.  The  graph  is  a  parabola. 


Figure  5  Maximum  reliability  vs.  relative  frequency 
of  defect  for  function  in  Figure  3 


For  r  -  .  2  (optimum  c  =  .8), 


H  *  .  04  -  .  2  +  1  »  .  84 
in 


The  minimum  value  of  It  (.75)  occurs  when  r  *  4-.  When  r  *  0  or  1,  R  *  1. 

m  am 

This  relationship  is,  in  a  sense,  in  accord  with  information  theory.  Uncertainty  is 

highest  for  equiprobable  events  as  when  Pr{D)  »  Pr[D')  (or  r  «  Then,  it  is  reason* 

able  to  expect  least  reliability  R  for  r  *  4  For  R  •  .5,  the  information  is  H  •  £p  log0— 

z  t  *P( 

*■  ( log2  ♦Hog2«i*l  «  1  bit.  For  r  «  .  2,  the  information  ie  .  2  log  +  .  8  log— ~  • 

»  .  4S  ♦  .  26  «  .  72  bits.  No  attempt  is  made  at  the  present  to  reiate  information  to  reli* 
ability  in  a  quantitative  manner. 


Operating  (iiir«rUrUtir  Cer«r 

In  signal  detection  theory,  receiver  operating  characteristic  (ROC)  curves 
are  drawn  to  show  the  relationships  between  Pr(yes  jS|  and  Pr  (yes  IS')  where  S  stands 


12. 

for  a  signal  being  present.  An  analogous  type  of  operating  characteristic  curve  can 
be  drawn  tor  the  present  data.  For  the  first  example: 

Pr[yes|D']  3  J^f0<x)dx  =  J*  < 2  -  2x)dx  *  2x  -  x2  |* 

=  2  -  1  -  (2c  -  c2)  =  1  -  2c  +  c2  -  (1  -  c)2 

PrfncjD]  =  /Cfi(x)dx  =  fc  2xdx  *  x2  |C  =  c2 
0  0  'o 

We  have  already  found  that: 

Pr[yes|D]  •  fl  fi  (x)dx  3  1  -  c2  and 
c 

Pr[no|D']  3/^f0(x)dx  =  2c  -  c2  . 

Note  that  Pr[ycs|D]  +  Pr[no|D]  =  (1  -  c2 )  +  c2  =  1  and 

Pr[yes|D']+  Pr[no|D'l  =  (1  -  c)2  (2c  -  c2)  -  1. 

Using  corresponding  values  of  Prfyes  jD]  and  Pr[yes  |D']  for  various  values  of  c,  the 
operating  characteristic  curve  can  be  obtained.  Table  1  gives  such  values. 

Table  1 

Conditional  Probabilities  for  Various  Values  of  Criterion  Level 


Pr[yesjD'J  Pr[yesjD) 


c 

*  d  -  c)2 

*  1  -  c2 

0 

i 

1 

.2 

.64 

.66 

.4 

.36 

.84 

.6 

.  16 

.64 

.8 

.04 

.36 

1.0 

0 

0 

A  plot  of  the  data  of  Table  1  is  presented  as  Figure  6  and  in  Figure  11  as  a  *  1. 
(The  similarity  in  the  form  of  this  curve  and  that  of  the  receiver  operating  character* 
istic  curves  of  signal  detection  theory  is  self  evident.  We  have  deliberately  called 
our  curves  operating  characteristic  curves  to  accentuate  the  similarity.) 


13. 


PrCmltfJ 

Figure  6  Operating  characteristic  curve  for  Example  1 


Such  a  curve  (Figure  6)  is  interpretable  in  several  ways.  First,  if  a  technician's 
performance  falls  below  the  points  of  the  curve,  he  is  not  performing  malfunction  di¬ 
agnosis  correctly.  If  his  performance  falls  along  the  transverse  line  shown,  he  is  per¬ 
forming  at  the  chance  level.  Here,  either  training  or  equipment  modification  (auto¬ 
matic  test  equipment)  might  be  indicated.  Finally,  for  a  fixed  value  of  c,  the  ordinate 
of  a  point  on  the  curve  gives  the  probability  of  a  malfunction  being  detected,  given  that 
a  malfunction  is  present;  the  abcissa  gives  the  corresponding  probability  that  a  mal¬ 
function  will  be  reported  when  none  exists,  in  fact. 


ill  Ciafidnrf  Diulrikii  i«« 

Let  us  assume  that  technician  confiden  t  nossesses  an  exponential  distribution. 
Exponential  confidence  frequency  curves  are  shown  ir  Figure  7. 


Figure  7  Frequency  functions,  with  (ft)  and  without  (f0)  defect 


14. 

The  function  for  no  defect  is  of  the  form  fo (x)  =  ke  A.  The  area  under  the 
curve  must  be  unity  and  accordingly: 


Z1 ke  Xdx  =  1 


-  ke  X  I1  =  -  k(e  -  1)  =  1 
'o 

e 


k  = 


e  ~  1 


=  1.532 


1-x 


,  ,  .  e  -x  e  „„„  1-x 

fo (x>  - - -  e  =  -  — -  =  ,o82e 

e  -  1  e  -  - 


The  function  when  there  is  i;  defect  is: 


fi(x)=-£— f  =  .  582eX. 
e  -  l 


The  various  conditional  probabilities  are: 
x 


Pr[yes  |D] 


Pr[no|D] 

Pr[noiD']  = 

and 


1  -2— dx  = 

-T-e*  I'  =  — ~(e-ec)  • 
e-1  'c  e-1 

e-e  =  1-e 

c  e- 1 

e-1  1-e"1 

.  el  ‘ X 

1  C  H  v  - 

1  i  -x  M  _  e  -e4 

e1  ”c  -  1 

,  ax  - 

C  G"1 

e- 1  ~  '  c  e-1 

e-1 

-  Pr[yes  |  T ] 

X  X 

=  rc® _ (jx  -  — |C  = 

c  i 

e  -  1 

o  e-1  e-1  'o 

e-1 

-  Pr[yes  |  T'] 

i-x  i-x 

=  rc® _ dx=  .1 _ Ic 

l-e’C 

Jo  e-1  0  e-1  1 o 

1  -  e‘i 

R  =  pr[D]Pr[yes  |D]+ Pr[D']Pr[no|D'] 


c-i 


+ 


a  -  r)cfrpr>- 


5. 


Op] t bun  Confidence  Level  for  Exponential  Case* 

If  one  wishes  to  determine  the  confidence  level  that  the  technician  should  adopt 
in  order  to  obtain  a  maximum  value  of  R,  the  technician  should  adjust  his  c  value  so 
that: 

fi  (c)  _  1-r 
fo(c)  r 


c  ,  -c 
e  , ,  el 


=  -fer-  *  «1C'‘  -  — 

e-1  e-1  el  L  r 


2c-l  =  ln<  — ) 
r 

2c  =  1  +  ln(  — ) 
r 


c  =  j  +  ?  In  <^-£) . 

r 


Sine i  0  <  c  <  1,  the  above  formula  for  c  holds  only  when 
1 


e+l  -  r  '  7TT  or  .  269  <  r  <  .731. 


1  6 

When  r  =  =  .  26S,  c  =  1,  and  when  r  =  -j^y  =  .731,  c  =  0  for  maximum  R.  For 

0  1 

these  values  of  r  and  c,  R  =  — rr  =  .731.  When  r<  —rr  c  remains  1  and  R  becomes 

m  e+l  e+l,  m 

0 

R„,  =  1-r.  When  r>  —tt,  c  remains  0  and  R  becomes  R  =  r. 
m  e+l  m 

The  plot  of  Rm  versus  r  is  shown  in  Figure  8,  where  the  curve  from  x  ■  — iy 


to  X  ~  is 


given  by  Rm  =  ^|y  £l  -  2e  2  V  r(l-r)j  .  At  r  =  .  5,  Rm  =  .  623. 


Figure  8  Maxi  mum  reliability  curve  for  exponential  frequency  functions 


16. 


Table  2  gives  values  of  Pr[yes  |D]  and  Pr[yes  |D']  for  drawing  the  operating 
characteristic  curve. 


Table  2 


Conditional  Probabilities  for  Various  Values  of  Criterion  Level 


for  Exponential  Frequency  Function 


c 

Pr[yes  |  D1  ] 

Pr[yes  [D] 

0 

1.000 

1.  000 

.2 

.713 

.  872 

.4 

.478 

.713 

.6 

.286 

.522 

.8 

.  129 

.  286 

1.0 

•  0 

0 

The  plot  of  Table  2  is  shown  in  Figure  11  as  "exp.  " 

Noraal  Confidence  Distribution 

The  case  in  which  confidence  is  normally  distributed  may  be  similarly 
developed.  Normal  confidence  frequency  curves  (er  =  1)  are  shown  in  Figure  9. 


Figure  9  Normal  frequency  function,  with  (fj )  and  without  (f0)  defect 


17 


When  no  defect  exists: 


k  -  — 
V  2tt 


.  A. 

-  f1  e  2  dx  =  1 

V  2ir  0 


k(.  34134)=  1 


k  =  -34134  =  2‘9296 


f0(x)  =  2.9296- 


fi(x)  =  2.9296 


VTi r 

e  2 

ViT 


,l  (*-i)2 

Pr[yes|D]  =  2.9296  I  — - - — dx 

J  c  V  2  tt 


when  x  =  c,  u  =  c-1 
x  =  1,  u  =  0 


.°  u2 


Pr[yes  J  D]  -2 


.  9296  f  — 

Jc-1  V  2w 

t-c  £ 

=  2.9296  1  . . 2 

Jo  V  2  tt 


let  u  =  x-1 
du  =  dx 


2.92961  — -  2  du 

Jo  v'  2w 


Pr[no|Dl  *  2.9296  1  — — —  dx 

Ji-cV  2tt 


Pr[yes  | D *]  =  2.9296  1  — -  dx 

J  c  vl7 

c  2 
/■  -  x 

Pr[no  ID1]  *  2.92961  -S.—T'  dx 


18. 


The  reliability  is: 


R  =  2.  9296 


Optiaun  Confidence  Level  for  Noraal  Distribution  Case 

For  a  relative  maximum  of  R,  the  technician  should  set  his  c  value  so  that: 

fi  (c)  _  W. 

fo(c)  "  r 

(c-i)a  cl  _  1-r 

e“  2  /e"  2  r 

c- J  1-r 

e  =  — 

c  -  i  =  in 

c  =  \  +  ln(~)  . 

Since  0  <  c  s  1,  this  formula  holds  only  when; 

_ 1 -  <  r  s  — 1  ■  or  .  3775  <  r  i  .6225. 

1  +  e+  ^  1  +  e  ^ 

When  r  =  .3775,  c  *  1,  and  when  r  ■  .6225,  c  ■  0  for  maximum  R.  For  these  values 
of  r  and  c.  R  =  .6225.  When  r  <  .3775,  c  remains  1  and  R  becomes  R  *  1-r. 
When  r  >  .6225,  c  remains  0  and  Rm  becomes  Rm  *  r.  Table  3  gives  Rm  for  values 
of  r  between  .  3775  and  .  6225. 


Table  3 


Optimum  Criterion  Level  and  Maximum  Reliability 
vs.  Relative  Frequency  of  Defects _ 


r 

c 

Rm 

.  3775 

1 

.  6225 

.  4 

.9055 

.  602 

.  5 

.  5 

.  561 

.  6 

.  0945 

.  602 

.  6225 

0 

.  6225 

The  plot  of  Table 

3  is  shown  as  Figure  10. 

Figure  10  Maximum  reliability  curve  for  "normal"  frequency  curve 

(<r  «  1) 


Table  4  gives  values  of  Pr(yes|D')  and  Pr(yesjD)  for  various  values  of 


Table  4 


Conditional  Probabilities  for  Various  Values 
of  Criterion  Level  for  Normal  Distribution 


Prfyes  |P*1 


Fr[yes  1  Pi 


0 

1.  000 

1.000 

2 

.  767 

.  845 

4 

.  544 

862 

6 

.  338 

.455 

a 

.  156 

.  232 

l 

0 

0 

The  graph  is  shown  in  Figure  1 1  as  9  •  1. 


Table  5  gives  k  for  various  values  of  <r.  As  «  approaches  0,  k  approaches  2. 

Table  5 

Normalizing  Constant  (k)  for  Various  Values 
of  Standard  Deviation _ _ 


k 

2 

5.  222 

1 

2.930 

1/2 

2.095 

1/3 

2.  005 

1/4 

2.  000 

When  a  defect  is  present. 


&C, . 


(x-1  V 

f,(x)  =  .e"  2ai 

a V  2ff 


!Hor*«l  Dt  Ml  r  »  bu  I  i«n  with  <r  =  .5 

As  an  example  of  a  value  of  <r  different  from  unity,  take  <r  «  .  5. 


f0(x)  =  4.  190  e 


•2x 


-2(x- 1)2 


f,(x)  =  4.  180  e 

The  confidence  frequency  curves  are  shown  in  Figure  12. 


Figure  12  Normal  frequency  function  (»  «  .  5), 
with  (f» )  and  without  Up)  defect 


ThU  represent*  a  considerable  improvement  over  *  •  1  shown  in  frigure  9. 

LIE  u* 

( 9  e*  1 

Pr(ye»|D)  «  k|  — _ ; — <*« 

)  p  »2* 


Pr{no)D] 


L  u* 
/■*  e~  T 

*  k|  — 
/LIE  y  2» 


■du 


Pr[yes 


-  — 

ID']  «  k  c-2-I 

J7^ 


Pr[no  ID']  =  k 


r-  u' 

/'  4? 

"  n 


du 


-du 


H  =  k 


v2ff 

--  u2 

7 


r  J  ~ — du  +  <l-r)f 

Jo  V2tt  J 


.  iL 

e'  2 


o  vTw" 


du 
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Optianm  (on  fi  tlmrr  l.evrl  fur  Narasl  Dial  ribution  with  f  *  lc»»f 

In  the  present  case,  for  a  relative  maximum  of  R,  the  technician  should  set  c 
such  that : 


This  reduces  to; 


ft(c)  1-r 

ToTcT  r 


.  <c-n> 
e  2*r2 

1-r 

_c^ 

r 

e 

-V  (c-j) 

t 

1-r 

r 

In  (“™“! 

e-i 

»*  In  (■ 

C  * 

!♦«* 

to 

r 


Since  0  <  c  *  l,  this  formula  holds  only  when: 

I  .  1 


<  r  * 


i.  »*■  i* 


24. 


When  it  =  i,  this  becomes  .  1 19  <  r  <  .  881.  For  r  <  .  1 19,  c  =1,  and  for  r  >  .881, 

c  =  0.  For  r  *  .  119,  and  r  *  .  881,  R  =.831.  For  <  .  1 19  (c  =  1),  R  «  1  -  r.  For 

m  m 

r  >  .  881  (c  =  0),  R  :  r. 

m 

Table  G  gives  R  for  values  of  r  between  .  119  and  .881. 
m 

Table  6 


Optimum  Criterion  Level  and  Maximum  Reliability 
vs.  Relative  Frequency  of  Defects 
(Normal  Distribution  with  a  =  .  5) 

r  c  Rm 


.  119 
.2 
.3 
.4 
.5 
.6 
.7 
.  8 
.881 


1 

.846 
.712 
.601 
.500 
.399 
.288 
.  154 
0 


.881 
.  813 
.  757 
.  725 
.715 
,  725 
.  757 
.  813 
.  881 


The  graph  is  shown  in  Figure  13. 


Figure  13  Maximum  reliability  curve  for  "normal"  frequency  function 

<*  »  .5) 


25. 


Table  7  gives  values  of  Pr(yes  j D ' )  and  Pr(yes  |D)  for  various  values  of  c. 


Table  7 


Conditional  Probabilities  for  Various  Values  of  Criterion  Level 
(Normal  Frequency  Function) 


c 

Prfyes  D'] 

Prfyes 

0 

1.000 

1. 000 

.  2 

.  674 

.933 

.  4 

.  396 

.  806 

.6 

.  194 

.  604 

8 

.  067 

.326 

1.0 

0 

0 

These  are  plotted  in  Figur  •?  11  as  cr  =  .  5. 

(irnrrt I  Straight  Liar  Caafidence  Di » t r i bu  l  i  on 

Let  us  now  consider  the  situation  in  which  a  more  general  straight  line  fre¬ 
quency  function  of  confidence  exists-  This  is  shown  in  Figure  14. 


Figure  14  Linear  frequency  function,  with  (f»l  and  without  (fj)  defect 


The  x  and  y  intercept*  of  y  •  f0Cx>  occur  at  a  and  o  respectively,  where  a  is  arbi- 
2 

trary  and  b  *  . 


26. 


f0(x)  =  -d 

a  a 

fi(x)  =  Xx+2(1.I) 

a2  a  a 

Pr[yes|D]  =  2{— )  -  (— )2 

S  at 

Pr[no ID]  --  1  - 

a  a 

Prfyes  j D ' ]  =  (i--)2 
a 

Pr[no  [D'l  = 

R  -  r[2(i^)-  . 

For  a  given  value  of  r,  the  maximum  R  occurs  when: 

c  =  a  +  (1  -  2a)r  and  is  given  by: 

Rm  =  j  2x*a  [( 1  -  a.)  -  ( 1  -  2a)r]  -  r[(l  -  a)  -  (1  -  2a>r]2 

+  (1  -  r)[a  +  (1  -  2a)r]fa  -  (1  -  2a)r]  j  . 

As  a r.  example,  let  a  =  .75: 

fo(x)  =  f<l-fx) 

,  .  .  _  32  8 

fi  (x)  ~  9  x  '  g  . 

The  optimum  c  level  exists  when: 


Values  of  for  various  values  of  r  a.-e  given  in  Table  8.  Th<_  graph  is 


shov.  n  in  Figure  15. 


Table  8 

Maximum  Reliability  for  Various  Values 
of  Relative  Frequency  of  Defects 


Figure  15  Maximum  reliability  curve  for  linear  frequency  function 

(a  = .75) 

Table  9  gives  values  of  Prfyes  | D ']  and  Pr[yes  [ D]  for  various  values  of  c. 

Table  9 

Conditional  Probabilities  for  Various  Values  of  Criterion  Level 
(Linear  Frequency  Function)  _ 


Prfyes  |D'l  Prfyes  |D] 


Prfyes  | D *1  Prfyes  |D) 


0 

1 

1 

.  6 

.  04 

.783 

.  1 

.751 

1 

.7 

.  0045 

.640 

.2 

.  537 

1 

.75 

0 

.556 

.  25 

.445 

1 

.8 

0 

.462 

.3 

.36 

.  996 

.9 

0 

.250 

.4 

.  218 

.960 

1.  0 

0 

0 

.5 

.  Ill 

.  889 

These  are  plotted  in  Figure  11  as  a  =  .75. 


28. 


Step  Function  Distribution 


Step  function  confidence  frequency  curves  as  shown  in  Figure  16. 


Figure  16  Step  function  frequency  curves 


Let  us  assume  that  the  steps  are  of  equal  width  and  the  vertical  increments 
are  equal.  Let  S^fx)  =  j ®  ^  x  »  6  ‘  *ts  6raPh  *s  shown  in  Figure  17. 

S8W 

\ 

8 

Figure  17  Step  function  curve 


It  is  possible  to  represent  fj  (x)  as  a  sum  of  step  functions, 
fj  (x)  »  k[S,  (x)  +  Si(x)  +  Sj(x)] 

i  2  4 

The  constant  k  must  be  such  that  the  area  under  the  curve  is  unity. 


A  =  kf1f1(x)dx  =  k/1[(Si(x)  +  S,(x)  +  S3(x)]dx 

0  0  4  2  4 

=  k(.  75  +  .  5  +  .  25]  =  1.5k 


29. 


1.5k  =  1 
.  2 

k  =  -  so: 

fi(x)=  |-[St(x)  +  Si(x)  +  S3(x)] 

O  4  2  4 

f0(x)=  2  -  f0(x)  =  2  -  “[Siix)  +  Sj{x)  +  S3(x)]  . 

O  4  2  4 


Let: 


I6(a,  bl  = 


Sr(x)dx  = 
b  6 


b-a  if  6  <  a  <  b 

b-6  if  *a  <  6  <  b 

0  if  a  <  b  <  6  . 


Then  : 


Pr[yes  j  D]  *  /*  Mx)dx>  |/JSi(x)  +  Si(x)  +  S|(x)]dx 
=  ![Ii(c,  1)  +  Ii(c,  1)  +  Ii  (c,  1)] 

O  4  2  4 

1  1  (  2 

Pr[yes  |D']  =  /cf0(x)dx  5  j  2  -  j]Si  (x)  +  Si(x)  +  Sa(x)] 
=  2  -  2c  -  4[Ii(c,  1)  +  Ii(c,  1)  +  I3(c,  1)]  . 

0  4  2  4 


dx 


Table  10  gives  values  of  Pr[yes  J  D]  and  Pr[yes  |  D ' ]  which  are  plotted  in  Figure  11 
as  curve  S.  The  graph  consists  of  four  straight  line  segments  (one  being  horizontal 


and  one  vertical). 
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Table  10 

Conditional  Probabilities  for  Various  Values  of  Criterion  Level 
_ _ (Step  Function) _ _ 


c 

Prfyes  D ' ] 

Pr[yes  ] 

0 

1 

1 

.  1 

.  8 

1 

.25 

.  5 

1 

.4 

.  3 

.  9 

.5 

.  167 

.  833 

.  8 

.  1 

.  7 

.75 

0 

.  5 

.9 

0 

.  2 

1.0 

0 

0 

Pr[no|D]  = 
Pr[nojD']  = 
R  = 


Simplifying  gives: 

R 


/Cfi(x)dx  =  -rtkiO,  c)  +  li(0,  c)  +  la (0,  c)] 

0  >5  4  2  4 


/f0(x)dx  =  2c  -4[Ii(0,  c)  +  li(0,  c)  +  li(0,  c)] 

0  o  4  2  4 


~r  [Ii(c,  1)  +  I|(c,  1)  +  Ii(c,  1)] 

+  (1-r)  |  2c  [li(0,  c)  +  I|(0,  c)  +  li(0,  c)]| 


r  +  2c( 1-r)  -  —  [li(0,  c)  +  ^(0,  c)  +  la(0,  c)j  . 

>5  4  2  4 


Due  to  the  nature  of  the  frequency  functions,  it  is  not  possible  to  solve  for 

optimum  c  for  a  given  r  value  from  the  equation  =  ““  •  Table  1 1  gives 

fi  (c)  1-r 

values  of  ■■■»—.  and  Table  12  gives  values  of  ■  ■  —  . 

fo(c)  6  r 

Table  11 


Frequency  Ratio  vs.  Optimum  Criterion  Level 
(Step  Function) 

fi  (f ) 

C  fo(c) 

0  to  .  25 
.  25  to  .  5 
.  5  to  .  75 
.  75  to  1 


0 

.5 

2 


. — — 


Table  12 
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Table  of  -i— vs.  r 
r 

(Step  Function) 

1  -r 


0  ® 

.  1  9 

.  2  4 

.3  2.33 

.4  1.5 

.5  1 

.6  .66? 

.7  .428 

.8  .25 

.9  .111 

1.0  0 


1  -  r  fi  (c) 

Suppose  r  =  .  3.  The  value  of  — —  is  2.  33.  The  closest  value  of  is  2 

and  c  is  between  .  5  and  .  75.  To  find  the  maximum  value  of  R  for  r  =  .  3,  the  value 

of  c  would  have  to  be  varied  between  .  5  and  .  75  and  R  calculated.  When  this  is  done, 

R  is  found  to  increase  linearly  from  .  833  to  .  85  as  c  increases  from  .  5  to  .  75.  Above 

.75,  it  drops  linearly  to  .70  for  c  =  1.  Therefore,  for  r  =  .3,  R^  =  .85. 


General  Case  of  Frequency  Functions 

The  general  case  of  arbitrary  frequency  functions  will  now  be  considered.  As 
an  illustration,  two  arbitrary  curves  were  drawn  with  a  French  curve.  Function 
values  for  fo  and  fi  were  read  and  used  to  compute  the  areas  under  the  curves  by 
Simpson's  one-third  rule.  The  ordinates  were  then  scaled  so  that  the  area  under 
each  curve  equaled  unity,  thus  giving  frequency  curves.  These  are  shown  in  Fig¬ 
ure  18.  Table  13  gives  the  function  values. 


Table  13 


X 

Normalized  Frequency  Function  Values  for 
the  Hypothetical  Example  of  Figure  18 

fi(x) 

fo<x) 

fi  (x) 

X 

fo(x) 

0 

1.  86 

0 

.55 

.95 

1.  18 

05 

1.  805 

0 

.60 

.  86 

1.  26 

10 

1.75 

0 

.65 

.74 

1.33 

15 

1.675 

.22 

.70 

.61 

1.40 

20 

1.62 

.41 

.75 

.50 

1.48 

25 

1.  53 

.57 

.80 

.37 

1.55 

30 

1.45 

.70 

.85 

.24 

1.625 

35 

1.  36 

.81 

.90 

.  130 

1.70 

40 

1.  265 

.92 

.  95 

0 

1.77 

45 

1.  17 

1.02 

1.00 

0 

1.85 

50 

1.  06 

1.  11 

Values  of  A0(x)  =  J*f0(x)dx  and  Ai  (x)  =  J*  f,  (x)dx  are  given  in  Table  14  and 
plotted  in  Figure  19. 


Table  14 


Areas  under  Frequency  Curves  from  0  to  x 


X 

A0(x) 

Ai  (x) 

0 

0 

0 

.  1 

.  1805 

0 

.2 

.348 

.022 

.3 

.501 

.078 

.4 

.637 

.  1595 

.5 

.754 

.261 

.6 

.849 

.379 

.7 

.923 

.512 

.8 

.973 

.660 

.9 

.998 

.823 

1.0 

1.  000 

1.000 

The  condition  for  optimum  value  of  c  is  still: 


pmppMgw-: 'jwpw. . j|i  >.'  !■  »•  £*i0& 
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1  *  r  7 

For  example,  let  r  -  .  3.  Then  — ; —  -  —  .  We  must  determine  a  value  of  x 

I*  M 

that  |Aj*|  =  j  .  This  can  be  done  by  drawing  a  template  with  two  lines  with 
7 

having  ratios  of  —  ,  as  in  Figure  20,  on  a  transparent  sheet. 


Figure  20  Template  with  lines  having  slopes  7:3  ratio 


such 

slopes 


7 

The  height  of  every  point  on  the  upper  line  is  ^  the  height  of  the  corresponding  point 
directly  below  it  on  the  lower  line.  The  x  axis  of  this  template  is  made  coincident 
with  the  x  axis  of  the  graphs  of  fo  and  ft  and  displaced  horizontally  so  that  line  1 
intersects  ft  and  line  0  intersects  f0  for  the  same  value  of  x.  This  occurs  when  x  = 
.707  so  that  c  1  .707  for  r  s  .  3.  The  maximum  value  of  R  can  now  be  computed  for 
r  =  3. 

l  •  T  !>1 

R  *  .3/  ft (x)dx  ♦  .7/  f0(x)dx 
m  .  jot  o 

Jl  ft  (x)dx  *  1  -  A,  (.707)  »  1  «  .520  «  .480 

•TOT 

•  TOT 

/  f0(x)dx  ■  .929 
0 

R  *  .  3(.  480)  ♦  .  7(.  929)  •  .794. 
m 

If  other  values  of  c  are  chosen  (with  r  fixed  at  .  3),  lower  values  of  R  result. 
Table  15  illustrates  this. 
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Table  15 

Reliability  vs.  Criterion  Level  for 


Constant  Frequency  of  Defect 


c 

R 

.5 

.75 

.6 

.78 

.7 

.793 

.8 

.783 

.9 

.75 

l 

-  Aq(c) 

Pr[yes  |D']  = 

/c  f0(x)dx  =  1 

l 

-  Ai  (c) 

Pr[yeslD]  - 

f  fifxldx  *  1 
c 

Table  IS  gives  values  of  Pr[yes|D']  and  PrtyeajD}. 


Table  16 


Conditional 

Probabilities  for  Various  Values 
of  Criterion  Level 

c 

Prlves  ID’] 

PrLyea  j  D] 

0 

1 

1 

.2 

.652 

.878 

.4 

.363 

.  o41 

.6 

.  151 

.621 

.8 

,027 

.340 

1.0 

0 

0 

The  curve  is  so  close  to  that  labeled  a  ■  1,  that  it  wa*»  not  plotted. 
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Discrete  Distribution  of  Confidence 


In  the  Naval  operational  situation,  only  a  few  levels  of  individual  technician 
confidence  are  probably  identifiable.  When  only  a  finite  number  of  confidence  values 
are  involved,  they  can  be  represented  as  shown  in  Figure  21  and  22. 


Figure  21  Probability  distribution 
when  a  defect  exists 


Figure  22  Probability  distribution 
when  there  is  no  defect 


The  vertical  lines  have  been  drawn  at  various  points  to  make  the  graphs  more  per¬ 
spicuous.  When  a  defect  exists,  the  probability  for  x  *  l  is  highest  (in  this  illustra¬ 
tion,  pi  ( 1)  *  .54).  When  there  is  no  defect,  the  probability  for  x  «  0  is  highest 
(po(0) 1  .6).  Values  for  this  arbitrary  example  are  shown  in  Table  17. 

Table  17 

Probability  Distribution 

*  in(x)  Potx) 

0  0  .6 

.23  .14  .3 

•  <8  .32  *  « 


In  the  discrete  case,  p0(x)  aud  pi(x)  correspond  to  fo(x)  and  fi(x)  respectively. 


2po(x)  and  2pi(x)  correspond  to  /fo(x)dx  and  SUM  respectively.  As  before; 


Pr[yes]  =  Pr[x  ^  c] 

1 

Pr[yesjD)  =  ^  pi  (x),  Pr[yes|D'] 

xic 


x<c  • 

Pr[no|D]  =  ^  pi(x),  Pr[no|D’] 


1 

S  Po(x) 

x>c 

x<c 

yi  po  (x> . 

x=0 


For  the  discrete  case,  when  c  is  a  value  for  which  po  or  is  defined,  then  it  makes 
a  difference  whether  Pr[yes]  -•  Prfxs  c]  or  Pr[x>c],  The  two  probabilities  differ  by 
p(c). 

1  x<l 

R  =  r^  pi  (x)  +  <  1  -  r)  V  po  (x) 

XiC  x=t) 


This  is  also  a  maximum  when; 

=  (It) 
r  * 

Consider  an  example  which  corresponds  to  the  continuous  example  shown  in  Figure  3. 

Table  18  gives  the  values  of  p0  andpi.  Table  18  also  presents  the  (cumulative)  dis- 

1  -  r 

tribution  and  ratios  of  pi  to  p2 .  For  r  =  .  8,  =  .  25.  From  the  table,  we  see 

that  the  nearest  value  of  the  ratio  is  so  c  should  lie  between  0  and  .  25. 

O 


£d£> 

P'J<c) 


Table  18 
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Probability  Distribution  and  Ratios 


-V 

Pi<x) 

Po(x) 

k 

jjPo(x) 

Pi  <x> 
Po(x) 

0 

0 

.4 

0 

4 

0 

.25 

.  1 

.3 

.  1 

7 

1/3 

.50 

.  2 

.  2 

.3 

9 

1 

.75 

.  3 

.  1 

.6 

1 

3 

1.0 

.4 

0 

1.0 

1 

CC 

Table  IS  gives  ranges  of  optimum 

c  and  values  of  R 

m 

for  various  values 

Table  19 


Range  of  Optimum  Criterion  Level  and  Maximum  Reliability 
_ _ vs.  Relative  Frequency  of  .Defects _ 


0 

.75-1 

1 

.2 

.75-1 

.88 

.3 

.5  -  .75 

.84 

.4 

.5  -  .75 

.82 

.5 

.5 

.80 

.6 

.25  -  .5 

.82 

.7 

.25  -  ,5 

.84 

.8 

0  -  .25 

.88 

1.0 

0  -  .25 

1 

The  values  of  were  obtained  from  Table  20,  which  gives  values  of  R  for 
various  values  of  c  and  r. 


Table  20 


Reliability  and  Conditional  Probabilities  vs.  Criterion  Level 


c  j 

Prlyes  |DJ 

£  Pi  <x) 
x>C 

x<c 

£  p  w 

r  =  .2 

R 

r  «  .3 

*1 

II 

4*> 

II 

cn 

Pr[yes  |D’j 

1 

.4 

.52 

.58 

.64 

.7 

.  6 

.2 

1 

.  4 

.52 

.58 

.64 

.7 

.6 

.4 

*.9 

.  7 

.74 

.76 

.78 

.8 

.3 

.5 

.9 

.7 

.  74 

.76 

.78 

.8 

.3 

.6 

.7 

.9 

.86 

.84 

.82 

.  8 

.  1 

.8 

.4 

1 

.  88 

.  82 

.76 

.7 

0 

1.0 

.4 

1 

.  88 

.82 

.76 

.7 

0 
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The  operating  characteristic  curve  has  four  points:  (.6,  1),  (.3,  .9),  (.  1,  .  7), 

and  (0,  .  4).  The  R  curve  has  a  minimum  of  .  8  for  r  =  .  5. 
m 

Double  Criterion  Level  Detection  Model 

To  this  point,  we  have  discussed  a  detection  model  with  a  single  criterion 
level.  When  the  confidence,  x,  of  there  being  a  defect  is  greater  than  or  equal  to 
the  criterion  level,  c,  the  subject  will  say  "yes"  (there  is  a  defect).  When  x  ^  c, 
the  subject  will  say  "no.  " 

Now  consider  two  criterion  levels,  cj  and  C2 ,  where  ci  <  C2.  If  x<  ci , 
the  subject  will  say  "no.  "  If  x  ?  C2  ,  he  will  say  "yes.  "  If  cj  <  x<C2 ,  then  the  sub¬ 
ject  will  make  another  attempt  to  determine  the  status  of  the  equipment.  This  con¬ 
tinues  until  he  says  either  "yes"  or  "no.  "  This  type  of  procedure  has  also  been 
called  by  Swets  (1964)  "  sequential  observations.  "  A  third  possibility,  not  con¬ 
sidered  her^is  that  he  may  be  allowed  only  a  limited  number  of  trials.  If  not  able 
to  decide,  this  might  constitute  a  third  type  of  unr  eliable  operation  on  the  part  of 
the  subject,  the  first  two  types  occurring  when  subject  says  "no"  with  a  defect,  and 
"yes"  with  no  defect. 

Assume  that  the  curves  shown  in  Figure  23  represent  frequency  functions. 


Figure  23  Hypothetical  frequency  function,  with  (fi)  and  without  (fo )  defect 
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The  area  under  each  would  then  be  unity.  The  two  criterion  levels  are  represented 
by  cj  and  c2 .  The  area  under  a  curve  f  from  c2  to  1  [call  it  A(c2 ,  1)]  represents 
the  probability  (pi )  that  the  technician  will  say  "yes"  on  one  trial.  Similarly,  A(0,  c) 
is  the  probability  (P3)  of  saying  no  "and  is  represented  by  the  area  under  a  curve  f  from 
0  to  cj  on  one  trial.  The  probability  (p2 )  of  being  undecided  on  a  single  trial  is  the 
area  under  a  curve  f  from  ci  to  c2 .  The  trials  are  considered  independent.  The 
probability  tree  in  Figure  24  shows  only  three  stages.  It  holds  for  either  a  defect  (D) 
or  no  defect  (D1). 


Y  Y  Y 


Pr[yes] 

Pr[no] 

Specifically: 


Figure  24  Probability  tree 

Pi  +  P2Pi  +  pipi  +  p|pi  ...  =  —Pi - 

1  -  p2 

P3  +  Pa  P3  +  pip3+  pips  ...  =  -r22 - 

1  -  Pa 

Prfyes  |DJ  =  — -■  Ai(cii  1) 

Ai  (c2 ,  1)  +  Ai  (0,  cj ) 

PrlnolD’]  =  Aaj°.  Cj> -  , 

Aq(c2,  1)  +  A0{0,  cj) 


Finally,  reliability  is: 


—PL 
Pi  +Ps 

—Pi.. 

P1+P3 


R  =  rPr[yes|D]  +  (1  -  r)Pr[no  |D’] 

=  rAi<c^  ^  +  (1-r)  An(0.  ci) _ 

Ai(c2,  1)  +  Ai(0,  ci)  A0(c2,  1)  +  A0(0,  ci)  * 
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The  expected  number  of  trials  for  a  decision  (yes  or  no),  if  up  to  n  trials  are 
allowed,  was  derived  previously  (Siegel  &  Miehle,  1967)  as; 


n  =  1  '  f1  +nO-P2>]p2 


n 

t  npa 


lim  En  = 


1 


1  -  P2  A(C2,  1)  +  A(0,  Cl)  • 


This  holds  for  both  the  cases  in  which  a  defect  or  no  defect  exists.  The  case  where 
an  unlimited  number  of  trials  are  allowed  therefore  is; 

^  r  ,  1  -  r 


Ai  (C2  ,  1)  +  Ai(0.  Cl)  Ao(c2,  1)  +Ao(0,  Cl)  ‘ 


The  increased  reliability  obtained  by  repeated  trials  is  obtained  at  the  expense  of 
increased  time  and  effort. 

As  an  illustration,  consider  the  linear  frequency  functions  of  Figure  3. 


Ai(C2,  1) 
Ai(0,  ci) 
.  A0(C2  ,  1) 
Aq(0»  Ci) 


i  j  l 

f  fi  (x)dx  =  /  2xdx  =  x2  1  =  1  -  C2 

C2  C2  C2 


Cl 

2  I  2 

X  =  Cf 

'o 


1  I  1 

/  f0(x)dx  =  /  (2  -  2x)dx  =  2x  -  xa| 

Cj  C2  C2 

Cl 

2x  -  x2  I  =  2C1  -  Cl 
'o 


Let  d  =  cj  -  ci 


R 


1  -  C2+  C? 


(1  -  r)(2c,  -  cU 


1  -  2cz  +  c2  +  2ci  -  ci 


1  -  2C2  +  C2 


*  jj. 

n 

■I 


r(l  -  ca) 


1  -  d(ci+  C2  ) 


(1  -  r)ci (2  -  ci ) 

1  +  d(ci  +  C2 )  ■  2d 


1  -  d(cj+  C2) 


_ 1  -  r _ 

1  -  2d  +d(ci  +  C2 ) 


E  is  plotted  as  a  function  of  c  = 


-  ci  +  c2 


for  r  =  .  3,  .5  for  d  =  .  2,  .4,  .6 


(Figure  25).  R  is  plotted  as  a  function  of  c  for  d  =  0,  .2,  .4,  .6  for  r  =  .  3  in 
Figure  26,  and  for  r  =  . 5  in  Figure  27. 

It  can  be  seen  that  the  optimum  value  of  c  is  .5  for  r  =  .  5  and  that  the  max¬ 
imum  value  of  R  increases  as  the  separation  (d)  between  the  criterion  levels  increases 
For  r  =  .3,  the  optimum  value  of  c  decreases  as  d  increases.  The  maximum  values 


(Rm)  of  R  increase  vith  increasing  d  and  are  higher  than  those  for  r  =  .  5  as  ex¬ 


pected. 


The  curves  of  for  d  >  .  2  have  relative  minima  whose  location  shifts  to 


the  left  for  increasing  d,  but  they  do  not  coincide  (except  for  r  =  .5)  with  the  max¬ 
ima  of  R. 


Table  21  gives  values  of  optimum  c,  Rm,  and  for  various  values  of  d. 


when  r  =  .  3  and  .  5. 


Table  21 


Optimum  Average  Criterion  Level,  Maximum  Reliability,  and 
Expected  Number  of  Trials  vs.  Separation  (d)  of  Criterion  Levels 


r  =  .  3 


r  * .  5 


.  7V1'. 

d 

c 

R 

m 

E 

00 

c 

R 

m 

E 

CO 

3? 

0 

.7 

.79 

1 

.5 

.75 

1 

m 

.2 

.63 

.831 

1. 22 

.5 

.  80 

1.25 

m 

.4 

.59 

.  868 

1.61 

.5 

.  85 

1.67 

S 

.6 

.54 

.910 

2.41 

.5 

.90 

2.5 

I 

.2  .3  A  .5  .6  .7  .8  .9  1.0 

C 


'e  26  Reliability  vs.  average  criterion  level 
(when  r  =  .  3) 


inns 


CHAPTER  III 
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DISCUSSION,  SUMMARY,  AND  CONCLUSION 

Discussion 

In  the  present  report,  reliability  and  operating  characteristic  curves  were 
considered  for  various  types  of  "confidence  of  the  existence  of  a  defect"  distribu¬ 
tions.  This  is  an  adaptation  of  the  method  used  by  Tanner  and  Swets  (1954),  where 
stimulus  is  replaced  by  confidence.  If  the  distributions  for  defect  and  no  defect,  are 
symmetrical  about  x  =  j,  then  the  maximum  reliability  (Rm)  is  lowest  when  a  defect 
occurs  with  probability  of  f.  This  is  the  situation  of  maximum  uncertainty.  For  all 
continuous  distributions  considered,  the  maximum  reliability  attainable  equals  1  when 
a  defect  is  never  present  (r  =  0)  and  when  a  defect  is  always  present  (r  ~  1).  The 

lowest  value  of  R  (at  r  -  .  5)  depends  on  the  distributions.  The  smaller  the  com- 
m 

mon  area  under  both  curves  (D  and  D'),  the  larger  the  Rm  values  are  (see  Table  22). 

When  the  curves  do  not  intersect,  R  is  1  for  all  values  of  r.  When  *•  is  low,  the  opti- 

m 

mum  criterion  level  (c)  is  high,  and  when  r  is  high,  c  is  low.  From  the  point  at 

which  c  first  becomes  0,  the  plot  of  P.  vs.  r  becomes  a  straight  line  (R  =  r).  From 

the  point  where  c  first  becomes  1,  R  vs.  r  becomes  the  straight  line  R  *  1  -  r. 

m 

Table  22 

Comparison  of  Common  Area  under  f»  and  f0  Curves 
with  Maximum  Reliability  for  r  *  ,  5 

Case  Area  Rm  for  r  «  .  5 


Non-intersecting 

0 

1 

Straight  line,  a  B  .  75 

.  222 

.889 

Straight  line,  a  =  1 

.  5 

.75 

Normal,  j  s  i 

.57 

.715 

Exponential 

.755 

.623 

Normal,  o  *  1 

.878 

.361 
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A  similar  progression  holds  for  the  operating  characteristic  curves.  As  the 
common  area  under  the  distribution  curves  decreases,  the  operating  characteristic 
curves  move  closer  to  the  left  and  upper  boundaries. 

The  reliability  value  (R)  may  be  improved  in  two  ways.  The  first  is  to  mini¬ 
mize  or  eliminate  the  intersection  of  the  frequency  curves,  as  indicated  by  Table  22. 
When  the  curves  are  non-intersecting,  the  maximum  attainable  reliability  is  1. 

This  reliability  is  obtained  when  the  confidence  criterion  level  is  properly  set.  One 
method  for  achieving  this  level  is  to  design  the  equipment  so  that  the  existence  of  a 
defect  is  more  obvious.  This  will  separate  curves  fj  and  f0,  making  the  setting  of 
a  criterion  level  less  critical.  With  non-intersecting  curves,  the  optimum  criterion 
level  is  anywhere  between  the  x  intercepts  of  the  curves. 

With  a  given  pair  of  frequency  curves,  the  reliability  may  also  be  improved 
by  training  the  technician  so  that  he  sets  his  criterion  level  towards  the  optimum 
value.  This  can  be  accomplished  by  providing  the  technician  with  feedback  of  in¬ 
formation  about  his  performance  or  possibly  through  programmed  instructional 
techniques. 
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Sti aiary  and  Conclusion 

The  present  report  has  attempted  to  point  out  the  relationship  between  the 
subjective  criterion  level  that  a  maintenance  technician  sets  or  adapts  for  accept¬ 
ing  or  rejecting  the  hypothesis  that  a  malfunction  exists  and  the  probability  of  his 
acceptance  or  rejection  of  the  hypothesis  being  correct.  The  logic  employed  is 
similar  to  that  used  in  signal  detection  tneory,  and  operating  characteristic  curves 
were  similarly  derived.  These  may  be  interpreted  in  much  the  same  way  as  the 
receiver  operator  curves  derive  !  in  the  psychophysical  investigation  of  signal  de¬ 
tection.  The  relationships  were  derived  for  a  number  of  continuous  distributions 
of  confidence  and  for  the  discrete  case.  Both  the  single  and  double  criterion  in¬ 
stances  were  considered.  It  is  believed  that  the  discrete  case  with  a  single  cri¬ 
terion  level  represents  the  most  useful  case,  from  the  point  of  vie*v  of  application. 
While  we  have  not  attempted  to  specify  a  method  for  measuring  a  technician's  con¬ 
fidence  criterion  behavior,  the  production  of  such  a  change,  and  the  consequent  in¬ 
crease  in  correct  Hypothesis  acceptance  and  false  hypothesis  rejection,  does  not 
seem  unattainable. 

The  gains  to  be  derived  from  the  production  of  such  a  shift  in  criterion  level 
are  considerable.  Consider  example  1  (Figure  3).  Assume  the  probability  of  a  de¬ 
fect  is  r  -  .  1.  The  optimum  criterion  level  is  c  *  1  -  r  ;  .  9.  At  that  level,  the 
maximum  possible  reliability  is  *r*  -  r+  l*.01-.l+l*  .91.  Suppose  the 
technician  actually  assumes  a  level  of  c  4  .5.  Then  his  reliability  is 

K  *  r  ♦  2c  -  cl  *  l re 
.75 

This  is  a  considerable  reduction  from  the  maximum  of  .91. 
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It  is  possible  that  a  programmed  learning  technique  or  program  could  be  de¬ 
veloped  to  allow  individual  technician  training  on  acceptance  criterion  level  setting. 
Thus,  the  acceptance  criterion  level  training  could  be  administered  quite  economi¬ 
cally,  with  the  usual  advantages  of  programmed  learning. 

Alternatively,  the  employment  of  the  acceptance  level  setting  behavior  of 
technicians  as  a  fleet  performance  criterion  seems  tenable.  Technicians  who  set 
their  criterion  of  acceptance  so  as  to  maximize  correct  hypothesis  and  rejection  are 
probably  superior  to  those  who  do  not.  Thus,  if  for  a  pair  of  technicians  and  a  given 
set  of  equipment  conditions,  technician  A  sets  his  criterion  level  less  close  to  the 
optimum  than  technician  B,  then  technician  B  may  be  said  to  be  superior.  Tech¬ 
niques  for  performing  such  measures  should  be  readily  derivable. 

Finally,  it  was  pointed  out  that  the  employment  of  the  operating  characteristic 
curve  possesses  implications  for  decisions  regarding  the  development  of  automatic 
test  equipment.  Test  equipment  would  be  built  for  those  situations  in  which  malfunc¬ 
tion  detection  deviates  significantly  from  the  optimum,  or  stated  alternatively,  where 
it  approaches  the  chance  level. 

In  conclusion,  it  seems  that  a  method  for  establishing  the  optimum  subjective 
criterion  of  acceptance  level  has  been  established  for  the  malfunction  recognition 
situation. 
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